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INTRODUCTION 



Recently, much concern has been expressed about the Importance of inservice 
preparation in science education along with the accountability for such activities. 
An attempt has been made to develop and try out a set of generalizable instruments 
which emphasize decision-oriented evaluation of an inservice teacher preparation 
prograi! in secox?dary school science. The context for this endeavor was embedded 
in a University/School cooperative teacher preparation model funded at the pre- 
service by Ind:JaP''^ HniMrersity-Bloomington and at the inservice level by the 

Natif'T ■•■^x ''he partners in this venture were the faculty of 

Indii\..' ♦'ion Department and the secondary school teachers 

of Mc ■•na 47401. 



• s packe : of instruments could be general- 
ize led and/or validated by others in similar 
sumir ip training/preparation situations. The 
in-:c*. V- ;C packet has been designed, is rather 
closely . ''ose / th ^ National Science Foundation (Imple- 
mentation Progx. /, E.S.L rograms, Urban Rural Programs, Bilingual 
Training Programs, Desegreg:.r ates, etc. Many times these federal agencies 
receive highly desirable training proposals which lack a solid evaluation plan; this 
paper/packet could be the basi3 for such a plan. 



RATIONALE 



When change in public schools and universities requires the implementation of 
new processes and practices, there is the need to first conduct descriptive evalua- 
tions designed to assess professional competence, values and attitudes, project 
strengths and weaknesses, and the goals of all groups involved. This University/ 
School cooperative project assessment mostly took on the form of "decision-oriented" 
or formative evaluation (Stuff lebeam, 1971). This type of evaluation used informa- 
tion collected from all levels for decision making', and desirable project improvement. 

The framework for assessing the degree to which the project hoped to succeed 
focused on the need for a variety of evaluation techniques. In order to succeed, 
the evaluation plan had to contain mechanisms for change and improvement based on 
a continuous flow of feedback data regarding the effectiveness of the University/ 
School cooperative project in achieving its soecified objectives. It was suggested, 
therefore, that the original evaluation plan ^FIGURE A) be composed of both formative 
and sunmative elements. The formative phase, which included needs-assessment , diag- 
nosis, and planning, was essentially concerned with assessing the degree to which 
each project component was operational and achieving its desired objectives. This 
phase of the evaluation assessed the degree to which the treatment aspects of the 
University/School cooperative model were functioning as defined in the original 
proposal or by the Project Director. A summative stage involved the overall evalua- 
tion of project success. This means of evaluation usually occurred at or near the 
termination of project key ctivities. Summative evaluation included measures of 
consultant ratings and student observations and attitudes, as well as indices 
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revealing the degree of successful -liapleinentation of project treatment components. 
If the evaluation plan were to be effective, it had to utilize a multifaceted 
approach. The evaluation effort, therefore, was designed to describe both the 
intent of the project and its methodology, to estimate its degree of implementa- 
tion, and to assess the degree to which it had succeeded in achieving its perform- 
ance goals. The evaluation plan provided m\ich more than terminal evaluation. At 
a time when program alterations could still be made, it provided data regarding: 
(1) whether or not the project was succeeding; and, (2) if the project was not 
succeeding, why this was so. 



PROCEDURES 



The inservica educati^oa strategy was that of a conventional six-week summer 
(1974) workshop and an academic year (1974-75) follow-up by way of two weekend 
mini workshops per semester, one demonstration day per semester, and bi-weekly 
consultant visits throughout the school year. luitially, a formative evaluation 
design (FIGURE A) was developed; this decision-making matrix w?:» a modification 
of the much-heralded CIPP Model of Stuff lebeam (Stuff lebeam, 1967). Based on the 
. needs of both the science education faculty and the inservice teachers, selected 
areas of concern were identified with appropriate instrumentation designed to meet 
these needs. Keeping the concern if decision making in mind with respect to on- 
going project modification and imp^. jvement , the instr' ' .^^nts took the forms of 
Likert scales, semantic differentials, SA-SD's, or open-ended response items. 



Description of Sub ^ jects 

The data necf^ 'sary to try out the in;9trument packet were obtained from 30 in- 
service teachers from a mid-eastern town (pop. est. 31,000). Although often con- 
fusing and misleading, and often misinterpreted, it is always interesting to examine 
the demographic dimensions of people, places, and things associated with public 
school classrooms. The mnnber of teachers per arbitrarily defined age groupings 
was 12 between twenty-one and thirty years, 11 between thirty-one and forty, 4 be- 
tween forty-one and fifty, 3 between fifcy-one and sixty, and none over sixty years 
of age. When considering the number of years of teaching experience, there were 
12 inso.rvice teachers with no experience to five years, 8 with six to ten years, 6 
with eleven to fifteen years, 3 with sixteen to twenty year and 1 with over twenty 
years of teaching experience. Turning tc* the current teaching assignment of the 
Involved inservice teachers, there were 12 teaching high school (9-12) science, 7 
teaching middle school (6-8) science, 3 teaching high school (9-12) math, 2 teach- 
ing nlddle school (6-8) math, 2 teaching elementary school (K-6) , and 2 labeling 
themselves as "other." In relation to the sex of the inservice teachers, there 
were 11 females and 19 males. When taking into account the type of school in which 
the participants taught, there were 29 from public schools and 1 f^'om a non-public 
school. And laatly, in view of the subject area preparations and/or cognate fields 
of the inservice teachers, there were 12 whose major interest was biology, 3 were 
in physics, 2 were in chemistry, 4 were in earth science, 3 were in mathematics, 4 
had a background in elementary education, and 2 identified themselves as "other." 

Data Analyses 

The information obtained was analyzed descriptively by way of means, standard 
deviations, modes, medians, and ranges along with frequencies and percentages. 
Aside from the quantitative descriptive differences as expressed by the instruments' 
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items or dependent variables, for further needed assurance, uhe data were also 
analyzed through the use of the nonparametric statistical treatment chi-square 
where a significant difference had to exhibit a probability that was less than 
.05 (p<.05). The differences determined by the use of chi-square were delineated 
by the demographic dimensions or independent variables discussed above. Quantitat- 
ive findings in many instances were supplemented by inservice teacher open-end 
commentary as it pertained to a given instiniment. The findings were used as the 
basis for rational project decision-making; the intent was not to conduct an 
elaborate inferential research study. Generally speaking, these conventional de- 
scriptive statistics helped yield desirable information; these ratings aided 
immensely with respect to decision-making and giving the project direction. 



SUMMER WORKSHOP 



The essence the workshop program focused on the development of specific skills 
needed for implementing modem secondary school science curricula. These are skills 
necessary for systematic observations and the utilization of data collection pro- 
cedures needed for local program evaluation, and skills in process interaction nec- 
essary for the development of long-lasting and productive interpersonal relation- 
ships. The remainder of this section will focus on a brief discussion of the in- 
struments developed and employed during the six-week summer workshop. The focus 
will not be on a discussion of inferences and conclusions associated with the collec- 
ted data. 



Self-Asseasment as a Teacher 

In order to obtain a baseline on how the insei'vice teachers perceived themselves 
as functioning in the classroom and in the secondary school as a whole, INSTRUMENT 1 
was constructed from several rating scales used by school administrators for per- 
formance evaluation and by universities for admissions into graduate study. The 
"Perceptions of Self as a Teacher" (JNSTRUUENT 1) was administered as a pretest dur- 
ing June 1974 on the first day of the workshop, and was re-administered in a 
posttest situation during the last academic-year follow-up session in April 1975. 
Other possible uses for INSTRUMENT 1, upon modification, could be for screening and 
selecting participants. Ratings might be obtained from building principals, depart- 
ment chairpersons, or former professors of the inservice teachers. 



Evaluation of Consultant Performance 

Usually during an externally funded workshop, the services of consultants are 
procured to perform some type of instructional function. Many times the consultants 
are reimbursed by way of honoraria. From both participant response and expenditure 
accountability standpoints, it might be wise to evaluate consultant performance. 
During the workshop four consultants were brought in for half -day presentations and 
interactions. Two were science educators, one a geophysicist and the other a geneticist. 
The participants were asked to respond to INSTRUMENT 2 on the day following the con- 
sultant visit. "Workshop Guest Speaker Evaluation" (I^/STRUME^^^ 2) is a collection 
of modified items coming from a school of education and the university forms used 
for evaluating instruction. 
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INSTRUMENT 1 

PERCEPTIONS OF SELF AS A TEACHER 



Pleaae rate yourself as a teacher on each of the items below. 
Circle the number of the rating which you feel is most appropriate. 
Please make comments on any item that you wish. 



Your Enthusiasm for and Interest in Teaching as a Career: 
12 3 4 

poor fair average excellent 

Your Potential All-Round Ability and Effectiveness as a Teacher: 
12 3 4 

poor fair average excellent 

Your Capacity for Improving Your Teaching Ability: 

12 3 4 

fair average excellent 



truly 
exceptional 



truly 
exceptional 



poor 



truly 
exceptional 



Your Interest in Improving Subject Matter Knowledge: 

12 3 4 

poor fair average excellent 



truly 
exceptional 



Your Capacity to Develop Student/Pupil Interest and Understanding: 

1 2 3 4 5 



poor 



fair 



average 



excellent 



truly 
exceptional 



Your Willingness to Cooperate with Colleagues » Administrators and Students: 

1 2 3 4 5 

poor fair average excellent truly 

exceptional 



Your Receptiveness to Constructive Suggestions from Others 
12 3 4 

average 



decicv^ily 
deficient 



distinctly 
below 
average 



distinctly 
above 
average 



exceptional 
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TWSTRUMEWr ; {Cont'd) 



8. Extent to Which You Can Be Relied Upon: 

1 2 3 4 5 

completely questionable dependable superior coinpontely 

untrustworthy at times dependability trustworthy 

9. Extent to Which You Have Been Conscientious In the Performance of Your Tasks: 



usually 
negligent 



Indifferent 



3 

average 



generally diligent 
and faithful 



10. Your Capacity to Work on Taaks with Other People: 
1 2 3 

much out sporadic average 

of gear 



generally 
cooperative 



supremely 
5 

supreme 



11. Your Interest In Improving the Science Program of Your School: 
12 3 4 

poor fair average excellent 



truly 
exceptional 



12. Your Interest In Improving the Entire Educational Program In the School: 



1 

poor 



2 

fair 



13. Your Intellectual Curiosity: 
1 2 



decidedly 
deficient 



distinctly 
below 
average 



3 

average 
3 

average 



4 

excellent 



truly 
exceptional 



distinctly 
above 
average 



exceptional 



14. Your Power of Grasping Ideas: 
1 2 



decidedly 
deficient 



distinctly 
below 
average 



3 

average 



distinctly 
above 
average 



exceptional 
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TNSTRUMEhrr 1 [Cont'd] 



15. Your Ability to Work Independently as a Student: 



vholly 
dependent 



needs much 
guidance 



16. 



Your Originality in Thinking: 
1 2 



decidedly 
deficient 



distinctly 
below 
average 



average 
Independence 



3 

average 



17. Your Personality and Initiative: 

12 3 
displeasing color lesj average 

18. General Rating of Yourself aa a Teacher: 

1 2 3 

poor fair average 



needs little 
guidance 



distinctly 
above 
average 



4 

pleasing 
4 

excellent 



independent 



exceptional 



extremely 
appealing 



truly 
exceptional 



Key to Items Above 

( ) 



COMMENTS 



( ) 



( ) 



( ) 



( ) 



ERIC 



2 



-9- 

JNSTRUMENT 2 
WORKSHOP GUEST SPEAKER EVALUATION 

Speaker: 

We vould like you to help us evaluate this guest speaker. The following 
statements relate to the quality of the speaker's presentation. Please indi* 
cate your agreement or disagreement with each statement by circling the appro- 
priate letters: sa ^strongly agree ; a-agree; u- undecided ; d »dis agree ; 
sd "Strongly disagree . 

1. The speaker had a thorough knowledge of the topic. sa a u d sd 

2. The speaker held my interest. sa a u d sd 

3. The speaker's voice was pleasant. sa & u d sd 

4. I would encourage my friends to hear this speaker. sa a u d sd 

5. The speaker was enthusiastic about the topic of his presentation, sa a u d sd 

6. The presentation was easy to understand. sa a u d sd 

7. The presentation was well organized. sa a u d sd 

8. The topic of the presentation was relevant to my interests. sa a u d sd 
9* I would like to hear this speaker again. sa a u d sd 
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Intrapersonal Dimensions 



One of the main themes of the workshop was a focus on the affective domain. 
The Inservice teachers Int eracted In non— threatening sessions and hopefully devel- 
oped the Interpersonal relationships necessary for continuous long term cooperation. 
The affective dimensions were treated directly In a few sessions. Developing the 
appropriate cooperative tone was so essential that attempts were made to Insure that 
this goal received constant attention in planning, implementing, and evaluating the 
program. In lieu of asking the participants to write a paragraph on "how they felt" 
periodically, INSTRUMENT 3 was developed with the aid of several textbook chapters 
on semantic differentials (Kerlinger, 1964; Oppenheim, 1966; and Tuckman, 1972). 
"Affective-Oriented Perceptions of Workshop Activities" (INSTRUMENT 3) was admin- 
istered during the last day of the second week, fourth week, and sixth week. The 
responses were processed, computed, and tabulated along four generalized dimensions 
and with respect to the 40 specific dimensions. 



During the six-week workshop experience, much of the participant work time was 
spent in small groups of three to four teachers. Since so much of the total work- 
shop time was allocated to participant group work, it was felt that an evaluation 
of this type of activity was appropriate. INSTRUMENT 4 was developed as a result 
of dimensions identified by science education faculty, graduate assistants, and 
the inservice teachers. "Affective Dimensions of Group Functioning" (INSTRUMENT 4) 
was administered during the last week of the workshop experience. Teachers' reactions 
to this type of functioning were very favorable. 



This component of the overall assessment plan began to look at the efficacy of 
the workshop from a summative evaluation perspective. The insv r^ice teachers were 
asked to assess their achievement of a given workshop objective by rating it on a 
one (perceived negative) Jto five (perceived positive) continuum of a Likert-t3rpe 
scale (INSTRUMENT 5). Usually the first portion of an item triad surveyed the 
absence to the abundant presence of a given objective achievement; the second part 
of the triad usually focused on the quality of the activities engaged in (ill- 
chosen and ineffective to^ very effective activities) for achieving the objective; 
and the third segment of the triad usually dealt with the number of opportunities 
available to achieve the given objective (far too few opportunities t£ more than 
enough opportunities). Ample lined space was also provided for optional in- 
service teacher open-ended commentary following the item triad for each program 
objective. "Evaluation Workshop Objectives" (INSTRUMENT 5) was administered 
to the participants on the last day of the workshop. 

Open-Ended Workshop Evaluation ; 

It was the general consensus that the participants should be given the oppor- 
tunity to express themselves with respect to making judgmental comments on any 
aspect of the workshop. Stake (1970) advocated that judgment data should be part 
of any evaluation scheme. The term judgment (Stake, 1970) has been used in a broad 
sense to include statements of priorities, values, opinions, and attitudes. Judgment 
data provided critically important information about workshop functioning because 
they represented what individuals perceived as happening. These perceptions may 



Participant Group Functioning 



Workshop Objectives' Evaluation 
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WSTRUMEhlT 3 

AFFECTIVE-ORIENTED PERCEPTIONS OF WORKSHOP ACTIVITIES 

Date: 

Please rate the individual or relationship whose name appears at the top of each 
series of items using the scales below the item stem (in capital letters). As an 
example, if you feel that the individual or relGf...>nBhip stated in the item stem 
is best characterized at either end of the continuom or somewhere between, circle 
the number which best describes what the individual or situation stated means to 
you. 



THE GENERAL ATMOSPHERE OF YOUR WORKSHOP EXPERIENCE HAS BEEN: 



1. 


Happy 


1 


2 


3 


4 


5 


6 


7 


Unhappy 


2. 


Cold 


1 


2 


3 


4 


5 


6 


7 


Warm 


3. 


Simple 


1 


2 


3 


<+ 


5 


6 


7 


Complicated 


4. 


Re laxed 


1 


2 


3 


4 


5 


6 


7 


Tense 


5. 


Pessimistic 


1 


2 


3 


4 


3 


6 


7 


Optimistic 


6. 


Emotional 


1 


2 


3 


4 


5 


6 


7 


Unemotional 


7. 


Democratic 


1 


2 


3 


4 


5 


6 


7 


Authoritarian 


8. 


Satisfying 


1 


2 


3 


4 


5 


6 


7 


Dissatisfying 


9. 


Competitive 


1 


2 


3 


4 


5 


6 


7 


Cooperative 


10. 


Dynamic 


1 


2 


3 


4 


5 


6 


7 


Static 




HOW WOULD YOU CHARACTERIZE 


YOURSELF DURING 


THE WORKSHOP 


EXPERIENCE: 


11. 


Leader 


1 


2 


3 


4 


5 


6 


7 


Follower 


12. 


Moody 


1 


2 


3 


4 


5 


6 


7 


Even disposition 


13. 


Dependent 


1 


2 


3 


4 


5 


6 


7 


Independent 


14. 


Active 


1 


2 


3 


4 


5 


6 


7 


Passive 


15. 


Outgoing 


1 


2 


3 


4 


5 


6 


7 


Shy 


16. 


Vivacious 


1 


2 


3 


4 


5 


6 


7 


Quiet 


17. 


Controlled 


1 


2 


3 


4 


5 


6 


7 


Erratic 


18. 


Serene 


1 


2 


3 


4 


5 


6 


7 


Stormy 
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xrusting 


1 


I 


3 




5 


6 


27. 


Democratic 


1 


2 


3 


4 


5 


6 


28. 


Dangerous 


1 


2 


3 


4 


5 


6 


29. 


Happy 


1 


2 


3 


4 


5 


6 


30. 


Emotional 


1 


2 


3 


4 


5 


6 




HOW WOULD YOU 


DESCRIBE 


THE GENERAL 


DISPOSITION OF 


31. 


Happy 


1 


2 


3 


4 


5 


6 


32. 


Cold 


1 


2 


3 


4 


5 


6 


33. 


Simple 


1 


2 


3 


4 


5 


6 


34. 


Relaxed 


1 


2 


3 


4 


5 


6 


35. 


Accepting 


1 


2 


3 


4 


5 


6 


36. 


Pessimistic 


1 


2 


3 


4 


5 


6 


37. 


Emotional 


1 


2 


3 


4 


5 


6 


38. 


Democratic 


1 


2 


3 


4 


5 


6 


39. 


Satisfied 


1 


2 


3 


4 


5 


6 


40. 


Dynamic 


1 


2 


3 


4 


5 


6 



Suspicious 

Authoritarian 

Safe 

Unhappy 

Unemotional 

JRKSHOP STAFF ; 

Unhappy 

Warm 

Complex 

Tense 

Rejecting 

Optimistic 

Unemotional 

Authoritarian 

Dissatisfied 

Static 
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INSTRUHEirr 4 
AFFECTIVE DIMENSIONS OF Gt^C^'P FUNCTIONING 



Read each statement carefully. Then indicate whether you (a) agree , (pa) 
probably agree , (pd) probably disagree > or (d) disagree with each statement. 
Circle the letters to indicate your answers. 

1. The teachers in my group(s) worked well together. a pa pd d 



2. This workshop experience enabled me to get to know my fellow a pa pd d 

teachers better. 



3. I got several good ideas from the other teachers in my a pa pd d 

group (s) . 



4. Everyone in my group(s) made important contributions to a pa pd d 

the group (s') progress. 



5. I enjoyed working with teachers from neighboring schools. a pa pd d 



6. I think I would like to communicate more often with a pa pd d 

teachers from neighboring schools. 



7. I would be interested in attending regular (monthly, bi- a pa pd d 

monthly, semi-annually, etc.) meetings of local science 
and math teachers to discuss new classroom ideas. 



8. As a result of this workshop I feel that a closer a pa pd d 

bond has been established between the participating 
teachers . 



9. I would like to be involved in another workshop where a pa pd d 

I could work with teachers from neighboring schools. 
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IWSTRUMEWr 5 



EVALUATION OF WORKSHOP OBJECTIVES 



Based on your participation in this six-week summer program, to 
what degree do you feel you have achieved the following objectives: 

(A) Cirole the number which best shows your relative position 
or react ion on the continuous scale. 

(B) Comment (s) sections are optional. 



I. OBJECTIVE: Develop instruction and the activities that have boen 
relevant to the participants' needs: 



1. Very relevant 

2. Activities were 
ill-chosen and 
ineffective 

Comment _ 



Irrelevant 

Activities very 
effectively con- 
tributed 



II. OBJECTIVE: Develop new teaching skills and materials; 

3. Developed a very 12 3 4 5 

large number 

4- They will be very 12 3 4 5 

useful 



Developed none 
Probably useless 



5. Activities were 
ill-chosen and 
ineffective 



Comment (s) 



Activities very 
effectively con- 
tributed 



III. OBJECTIVE; Develop skills in the collection and recording of data from 
systematic observation of classroom lessons: 



6. Developed a great 
deal of skill 



Developed no 
skill 
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IWSTRUMEAfT 5 [Cont'd) 



7. Activities were 
ill~chosen and 
ineffective 

8. No opportunities 
were provided 



Comment (s) : 



Activities very 
effectively con- 
tributed 

More than enough 
opportunities were 
provided 



IV. OBJECTIVE: Develop skills in indirect conferencing techniques 

1 2 3 4 5 



9. Developed a great 
deal of skill 

10. Activities were 
ill-chosen and 
ineffective 

11. No opportunities 
were provided 



Developed no 
skill 

Activities very 
effectively con- 
tributed 

More than enough 
opportunities were 
provided 



Conmient(8) : 



V. OBJECTIVE: Develop a feeling of "fellowship" with the other inservice teachers: 



12. Developed a very 
strong feeling 

13. Activities facili- 
tated this 
development 



No such feeling 
was present 

Activities hindered 
this development 



Comment (s) : 



VI. OBJECTIVE: Develop better personal and working relationships with the science 
education staff: 

14 Developed excellent 12 3 4 5 Developed poor 

relationships relationships 
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15. Activities facili- Activities hindered 

tated this 1 2 3 4 5 this development 

development 

Comment (s): ^ 



OBJECTIVE: Develop skills for the implementation of nationally supported 
science curricula: 



16. Developed a very 
large number 



Developed none; 



17. Activities were 
ill-chosea and 
ineffective 

18. No opportunities 
were provided 



Comment (s) : 



4 
4 



Activities very 
effectively con- 
':rJ-r>uteQ 

V'iiTC thax^ enough 
o p p o r t ii n 1 1 ie s 
vjssre provided 



OBJECTIVE: Jevelop skills necessary for the individualization of instruction: 



19. Developed a very 
large number 



De\'e loped none 



20# Activities were 
ill-chosen and 
ineffective 



Activities very 
effectively con- 
tributed 



21. No opportunities 
were provided 



More than enough 
opportunities 
were provided 



Comment <s) : 
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not correspond to the intentions or perceptions of those who designed the works^^P' 
However, what participants perceive ^s happening represents reality to them, an^J 
this might be xzr more important for the evaluation of a workshop than xAiat is 
supposed to be happening. As a result of such thinking, I^/ST)?UME^^^ 6 was developed 
to capture the candid and sincere responses of the inservice teachers. "Open-En^^^ 
Evaluation of Workshop Activities" (INSTRUMENT 6) was administered to the particip^^ 
during the last day of the workshop. 



ACADEMIC YEAR I'OLLOW-UP 



The academic year follow-up program consisted of two mini workshops per setxi^^t^ 
During the fall semester of 1974 they were held in October and December, and duiri^S 
the spring semester of 1975 the mini workshops were conducted during February ati^ 
April. The real thrust of the academic year follow up, however, had been the bi^ 
weekly consultant visits to the involved secondary schools by a science educatic?^* ^ 
department staff member to help the implementing teac' "^rs and interact with seco^^^^ 
school students. Once again, in this section, the dit>course will focus on the iX^'^ 
struments developed and used during the academic year. The emphasis will not be 
a discussion of inferences and conclusions associated with the collected data. 



Classroom Science Teaching 

During the summer workshop, the inservice teachers received instruction and 
were provided practice in using systematic data collection techniques. The parti^" 
ipants were fairly well schooled with respect to the need to collec • observational 
data and became quite willing to allow data to be collected from their classroom^- 
During the first follow-up session in October the project staff and small groups 
of the teachers with the aid of a few unidentified dittoed evaluation scales de- 
veloped INSTRUMENT 7. The instrument was administered four times duriag the 
academic year in the months of October 1974, December 1974, February J.975, and 
April 1975. "Science Classroom Profile" (INSTRUMENT 7) was filled ou^: by the con~ 
sultant immediately after observing a t^ach 's classroom and interacting with sec- 
ondary school students. (There was only one science education staff person serving 
as the implementation consultant throughout the academic year.) The consultant 
agreed to use the same set of arbitrarily established criteria during each asses^"" 
ment process. 



Classroom Observation 

During the first follow-up session in. October, it was agreed that a second 
of classroom observational data be collected focusing more specifically on selected 
teacher behavior. Another set of information could also complement the first set- 
Reliability between both sets of data would also lend credence to the rating process 
of the visiting consultant from a qualitative/descriptive viewpoint. The project 
staff and a small group of teachers using several collected unidentified dittoed 
questionnaires developed INSTRUMENT S. This instrument was responded to by the 
consultant immediately following an observation/conference with a teacher and in-" 
teraction with selected secondary school students. Again, the consultant agreed to 
use the same set of arbitrarily established criteria during each assessment process 
using the "Classroom Observational Scale" (INSTRUMENT 8) during October 1974, 
December 1974, February 1975, and April 1975. 
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TWSTRUMEWT 6 



OPEN-ENDED EVALUATION OF WORKSHOP ACTIVITIES 



Workshop Evaluation : 

In th^ ^pace pKovldzd bzZovo and pz/ihap6 on th^ back 6^dz oi 
thl6 6hcct, we Would appx^clato. you^ commznt^ conce^n^ng the 4^x- 
u;eefe woxkihop. We axz ioxcQolng oun. u^ual chzckll6t ok quzitlonnalKd 
e^valuatlon technique In oKdc/i to g^ve mo^e vozlght to th^ 6^nt^fmnt6 
u)hA.ch you <Jee£ z^pzclally worthy oi ^eco^dlng. We a6k you to /teco^d 
youK XQ.(iomm^ndatlon6 ^o^ ^mp^ov^ng ifi^^ type o^ u)o^k6hop. We a£4o 
n.(Lqu(L6t that you Kcco^d what you thought voa^ paKt^culaKly u^^iul lo 
that th(L6C activities muZd 6e continued. Please ^eel i^ee also to 
note what you did not like. You may sign youn. sheet ox not, as you 
wish. 

Thank you. 



Verbatim Comments per Participant 



I thought the workshop was very useful because it provided time and materials 
to develop a specific project for use in the school next fall. I believe everyone 
came up with something of value. In working with other members from my school, I 
felt much was gained. On the other hand, isolation from other members was not really 
desirable. Some comment has already been made about supplies. On the last day one 
member I know wanted to dry-mount and laminate his whole project and there were no 
materials, either those that were ordered, or those that could be borrowed from the 
host school. I would like to have had more structure in the Institute; at least one 
organized lecture each day; five and one-half hours of time to work independently 
was much too long a block of time. In some cases the time was not used productively. 
It would have been helpful tc have had the Director on site during the Institute. 
The role of the graduate assistants was very ambiguous. Their help varied from being 
very useful to being useless. I think they did not conceive of their role in a 
clear-cut way. One really did not know whom to ask for a g^lven item. Some flexibility 
is necessary and good, but I believe they would have also benefited by a designated 
role. 
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IWSTRUMENT 7 

SCIENCE CLASSROOM PROFILE 



1. 
2. 



6. 



7. 



8. 



y. 



10. 



11. 



12, 



13. 





1 


2 


3 


4 


5 




Teacher performs 
demonstrations 












btudents work with laboratory 
activities 


Definitions formu- 
lated bv students 












Definition supplied by teacher 
or textbook 


Written tests only 
for evaluation 












Evaluation including observation 
and performance tests 


Emphasis upon how 
to Solve problems 












Emphasis upon how things are put 
together or operate 


Teacher's main task 
is to present infor- 
mation 












Teacher serves as a resource 
person 


C "i 71 CT 1 o o o o rmm ATI t* 

^juigxc aooignmcnu 
for entire class 












Widely varied assignments 


Direct students 
by providing 
answers 












Direct students by asking 
questions 


Students' data may 
lead to fleveTfll 
different conclu- 
sions 












Students' data leads to one correct 
conclusion 


Teacher knows 
answers to all 
problems 












Teacher may explore and learn with 
students 


Student activities 
aimed at verifying 

facts OT" ni"*! DP"! n1 PS 












Student activities used as a 
source of basic data 


Teacher allows time for 
students to discover 
their own errors — 
uses questions to point 
out possible problems 












Teacher points out students' 
errors as they occur 


Students capable of 
much independent 
activity 












Students require much direction 
and assistance 


Students explore ideas 
suggested by teacher 












Students have freedom to explore 
materials according to their own 
curiosity 
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INSTRUMENT 8 
CLASSROOM OBSERVATIONAL SCHEDULE 



A Often 

B = Sometimes 

C « Never 

D = Not Observed 



A B C D 



1. The teacher employs divergent rather than convergent 
questions to guide students. 



2. The students are allowed to work at various rates of 
speed. 



3- Students form the h3rpotheses for new problems 
rather than the teacher. 



4. Students state the conclusions when analyzing data 
rather than the teacher. 



5. The teacher acts as a guide rather than "teller" 
of science. 



6. The teacher is comfortable even when he/she does not 
know the answers to students' questions* 



7. The teacher is willing to explore with the students 
when she does not know the answer. 



8. The teacher takes time to listen, question and 
observe individual students. 



9. The teacher is alert to groups or individuals who 
are too frustrated to continue their work. 



10. The teacher will correct students by guiding rather 
than by telling the answer. 



H 
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Secondary School Students' Evaluation 

Many of the teachers and all of the department chairpersons felt, "as long 
as everything else was being evaluated," that an evaluation by the i;econdary stu- 
dents might be interesting and possibly yield Important classroom decision-making 
information. All teachers agreed, even a few hostile ones, to allow their stu- 
dents to rate them on selected teaching/learning dimensions. Early In the school 
year the consultant and a small group of teachers, with the aid of several un- 
identified dittoed scales and a few group identified items, constructed INSTRUMENT 
9. The consultant picked up the administered "Student Questionnaire" (IWSTRUMEWT 9) 
during his December 1974 and April 1975 visits to each Implementing classroom. It 
was also agreed upon that the participants/teachers would administer the "Student 
Questionnaire" (INSTRUMENT 9) to all of the sections he or she taught, which in 
most cases wre six classes. 



Inservice Teachers^ Supervisory Behavior 

One of the intents of th- proposed project was to prepare inservice teachers 
to function within a much larger institutional effort. The institutional commit- 
ment was an emerging preservice "Secondary Science Teacher Preparation Program" 
(SSTPP) which had been funded internally by the Division of Teacher Education. 
The inservice teachers received instruction and utilized indirect counseling and 
training techniques for use with preservice teachers who might be preparing and 
teaching lessons. The project participants were taught a model of supervision 
focusing on the "helping relationship" for use with preservice teachers. The In- 
service teachers were to direct preservice teachers into initial teaching experi- 
ences. Lessons to be taught by the preservice teachers were to be selected from 
materials associated with the nationally supported secondary science programs. 

In order to evaluate the achievement of this objective, INSTRUMENT 10 was 
constructed by modification of a scale developed by Blumberg and Amidon (1965). 
"Preservice Teacher's Perceptions of Inservice Teacher's Supervisory Behavior" 
(IWSTRUMEWT 10) was administered by the classroom teacher during a preservice 
teacher's field-based experience of which he/she was the supervisor. INSTRUMENT 
10 was collected by the consultant during his visits. 



CONCLUDING REMARKS 



The evaluation effort described in this paper has been an attempt to "get-at" 
what was happening by affording a glimpse at all perceived relevant aspects of 
the Inservice teacher preparation program. This "holistic" type of evaluation 
(Stake, 1972) involved the assessment of most of the intended goals, the Interactions 
that occurred, and the actual project outcomes. The understanding of an emerging 
inservice secondary science education program is probably proportional to the 
quality and quantity of the information concerning the program. In one sense, the 
ultimate evaluation might describe everything that happened, the situation In which 
it occurred, and the effects of each event. With information of this nature, the 
decision-makers are in a better position to establish causal linkages between events 
and outcomes. Program modifications based on these analyses will more often lead to 
program improvements. Of course, time and financial costs will limit the compre- 
hensiveness of any evaluation effort. As was the case in this endeavor, there were 
dozens of dimensions needing evaluation that were left untouched. However, if the 
evaluation is to be a true estimation of worth, it must proceed as far as possible 
in an attempt to capture the complexity of what is really happening in a program. 
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IHSTRUMEUT 9 
STUDENT QUESTIONNAIRE 




!• My teacher likes to have me ask questions. 



2. My teacher has activities that I can do. 



• 3. I take part in the discussions in class. 



4. My teacher provides extra materials in class. 



5. My teacher tries to make class interesting. 



6. I enjoy science class. 



7. }fy teacher provides group activities. 



8. My teacher gives me suggestions of activities 
that I can do at home. 



9, My teacher does the experiments in class. 



^10. I get to do experiments in class. 
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Slgnlflcance of Effort 

The early 1970*8 have provided evidence as to the Increasing need for a more 
intense involvement of teachers In the program development and evaluation of their 
own preparation. It appears reasonable to presume that preparation/training 
areas, such as program development, teaching methods, learning environments and * 
teaching materials associated with inservice education will continue to be 
priorities during the late 1970' s. The instruments described earlier have the 
potential for modification and tailoring to meet the evaluation needs of others In 
somewhat similar situations, either functioning with external or internal funding. 
And then there are those operating without extensive resources when developing an 
evaluation component for a proposal seeking external funds; this paper might be 
the basis for such a proposal section. Aside from the formative kinds of evaluation 
Information specifically collected for this particular project, consideration might 
be given to the increased validity of the instrumentation presented here by way 
of future refinements and replications. There al'::o exists the potential fur the 
development of a formative evaluation model which could provide for the exportation 
of processes and products resultant from a mutually developed set of needs and 

• values among public schools and college/universities. And lastly, the more recent 

approaches to evaluation appear highly appropriate for both the development of emergent 

^ inservice projects and the revision traditional inservice programs to make them 
more timely and effective. 
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